1. Field of the Invention
This invention relates to an image processing method and apparatus, wherein processing for suppressing noise and/or processing for enhancing a desired structure pattern is performed on an original image signal representing an original image. This invention also relates to a recording medium, on which a program for causing a computer to execute the image processing method has been recorded and from which the computer is capable of reading the program.
2. Description of the Prior Art
Techniques for obtaining an image signal, which represents an image, carrying out appropriate image processing on the image signal, and then reproducing a visible image by use of the processed image signal have heretofore been known in various fields. For example, in Japanese Unexamined Patent Publication No. 55(1980)-163772, the applicant proposed a method for performing enhancement processing in a frequency domain, such as unsharp masking processing, on an image signal, such that a visible radiation image may be obtained, which has good image quality and can serve as an effective tool in, particularly, the efficient and accurate diagnosis of an illness. With the processing in the frequency domain, an unsharp 1019-1033, July 1991.
The Laplacian pyramid technique has been proposed in, for example, Japanese Unexamined Patent Publication Nos. 5(1993)-244508, 6(1994)-96200, and 6(1994)-301766. With the proposed Laplacian pyramid technique, masking processing is performed on the original image by using a mask having characteristics such that it may be approximately represented by a Gaussian function. A sub-sampling operation is then performed on the resulting image in order to thin out the number of the picture elements to one half along each of two-dimensional directions of the array of the picture elements in the image, and an unsharp image having a size of one-fourth of the size of the original image is thereby obtained. Thereafter, a picture element having a value of 0 is inserted into each of the points on the unsharp image, which were eliminated during the sampling operation, and the image size is thereby restored to the original size. Masking processing is then performed on the thus obtained image by using the aforesaid mask, and an unsharp image is thereby obtained. The thus obtained unsharp image is subtracted from the original image, and a detail image of a predetermined frequency band of the original image is thereby obtained. This processing is iterated with respect to the obtained unsharp image, and N number of unsharp images having sizes of xc2xd2N of the size of the original image are thereby formed. As described above, the sampling operation is performed on the masking-processed signal is subtracted from an image signal representing an original image, the resulting difference value is multiplied by an enhancement coefficient, and the thus obtained product is added to the image signal. In this manner, predetermined frequency components in the image are enhanced.
Also, as techniques for processing an image signal, techniques referred to as multi-resolution transform techniques have been proposed. With the proposed multi-resolution transform techniques, an image is transformed into multi-resolution images, each of which is of one of a plurality of different frequency bands, and predetermined processing is performed on each of the images of the different frequency bands. Images obtained from the processing are then subjected to inverse multi-resolution transform, and a final processed image is thereby obtained. As the multi-resolution transform techniques, a wavelet transform technique, a Laplacian pyramid technique, and the like, have heretofore been known.
The wavelet transform technique has recently been developed as a frequency analysis method and has heretofore been applied to stereo pattern matching, signal compression, and the like. The wavelet transform technique is described in, for example, xe2x80x9cWavelets and Signal Processing,xe2x80x9d by Olivier Rioul and Martin Vetterli, IEEE SP Magazine, pp. 14-38, October 1991; and xe2x80x9cZero-Crossings of a Wavelet Transform,xe2x80x9d by Stephane Mallat, IEEE Transactions on Information Theory, Vol. 37, No. 4, pp. image, which has been obtained from the masking processing with the mask having the characteristics such that it may be approximately represented by the Gaussian function. Therefore, though the Gaussian filter is used actually, the same processed image as that obtained when a Laplacian filter is used is obtained. Also, in this manner, the images of low frequency bands, which have the sizes of xc2xd2N of the size of the original image are successively obtained from the image of the original image size. Therefore, the group of the images obtained as a result of the processing is referred to as the Laplacian pyramid.
The Laplacian pyramid technique is described in detail in, for example, xe2x80x9cFast Filter Transforms for Image Processingxe2x80x9d by Burt P. J., Computer Graphics and Image Processing, Vol. 16, pp. 20-51, 1981; xe2x80x9cFast Computation of the Difference of Low Pass Transformxe2x80x9d by Growley J. L., Stern R. M., IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 6, No. 2, March 1984; xe2x80x9cA Theory for Multiresolution Signal Decomposition; The Wavelet Representationxe2x80x9d by Mallat S. G., IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 11, No. 7, July 1989; xe2x80x9cImage Compression by Gabor Expansionxe2x80x9d by Ebrahimi T., Kunt M., Optical Engineering, Vol. 30, No. 7, pp. 873-880, July 1991; and xe2x80x9cMultiscale Image Contrast Amplificationxe2x80x9d by Pieter Vuylsteke, Emile Schoeters, SPIE, Vol. 2167, Image Processing (1994), pp. 551-560.
In a radiation image, quantum noise of radiation becomes perceptible at an image area, which corresponds to an area exposed to a low dose of radiation and which has a low image density. Therefore, various methods have been proposed, wherein multi-resolution transform is performed on an image signal, which represents a radiation image, with a technique, such as the wavelet transform, band-limited image signals falling within a plurality of different frequency bands are obtained from the multi-resolution transform, and processing for suppressing noise is performed on the band-limited image signals. The methods are disclosed in, for example, Japanese Unexamined Patent Publication Nos. 6(1994)-274615 and 9(1997)-212623.
For example, Japanese Unexamined Patent Publication No. 6(1994)-274615 discloses a method, comprising the steps of:
performing wavelet transform on an image signal by employing a second-order derivative of a smoothing function as a basic wavelet function, band-limited image signals falling within a plurality of different frequency bands being obtained from the wavelet transform,
in cases where image processing is performed on each of the band-limited image signals, detecting a point, at which a signal value of a frequency band lower by one stage than a desired frequency band is zero,
setting an enhancement coefficient such that an area in the vicinity of the detected zero point takes a value larger than the values of the other areas,
enhancing the band-limited image signal of the desired frequency band with the set enhancement coefficient, and
performing inverse wavelet transform on the thus processed band-limited image signal and the band-limited image signals, a final processed image signal being thereby obtained. Of a radiation image, major object image information is expressed in a comparatively low frequency band among the plurality of different frequency bands after the wavelet transform, and noise components are expressed in a comparatively high frequency band. Therefore, there is a strong probability that the zero point, at which the image signal of a low frequency band among the image signals falling within the plurality of different frequency bands takes a value of zero, will represent an area associated with an inflection point of the image signal representing the boundary between the major object and the other areas, i.e. an area associated with an edge area of the major object. Also, there is a strong probability that the zero point, at which the image signal of a high frequency band takes a value of zero, will represent an area associated with a noise component. Accordingly, the enhancement coefficient is set such that the value in the vicinity of the zero point of the image signal of a comparatively low frequency band may become large. Also, the image signal of a frequency band higher by one stage than the frequency band, from which the zero point has been detected, is multiplied by the thus set enhancement coefficient. As a result, in the image signal of the frequency band higher by one stage than the frequency band, from which the zero point has been detected, the area corresponding to the edge area of the major object can be enhanced. In this manner, an image signal can be obtained, in which only the area corresponding to the edge of the major object has been enhanced.
Also, Japanese Unexamined Patent Publication No. 9(1997)-212623 discloses a method, comprising the steps of:
performing wavelet transform on an image signal, band-limited image signals falling within a plurality of different frequency bands being obtained from the wavelet transform,
processing each of the band-limited image signals such that a signal value smaller than a predetermined threshold value is converted into zero, and
performing inverse wavelet transform on the band-limited image signals having been obtained from the processing, a final processed image signal being thereby obtained. With the disclosed method, the signal value corresponding to an area of a comparatively low image density, at which noise is perceptible, becomes zero, and therefore the area of a comparatively low image density, which may be regarded as noise in the image, can be rendered to take a value of zero. In this manner, noise components in the image can be eliminated.
In the method disclosed in Japanese Unexamined Patent Publication No. 6(1994)-274615, enhancement of the desired band-limited image signal is performed in accordance with the signal values of the frequency band lower by one stage than the desired frequency band. However, with the disclosed method, image information representing an object having a fine structure in the desired frequency band is not reflected in the band-limited image signal of the low frequency band, and therefore the fine structure cannot be enhanced. Accordingly, the problems occur in that, in the image represented by the final processed image signal, the image of the object having the fine structure becomes imperceptible.
Also, in the method disclosed in Japanese Unexamined Patent Publication No. 9(1997)-212623, the values of the band-limited image signal smaller than the predetermined value are converted into zero. Therefore, the problems occur in that the signal components, which represent a structure pattern in the image but have signal values smaller than the predetermined value, are regarded as being noise and eliminated from the processed image signal.
The primary object of the present invention is to provide an image processing method, wherein image processing is capable of being performed such that noise components contained in an image become imperceptible, and such that a structure pattern contained in the image becomes perceptible.
Another object of the present invention is to provide an apparatus for carrying out the image processing method.
The specific object of the present invention is to provide a recording medium, on which a program for causing a computer to execute the image processing method has been recorded and from which the computer is capable of reading the program.
The present invention provides a first image processing method, comprising the steps of:
i) forming band-limited image signals representing images, each of which is of one of a plurality of different frequency bands, from an original image signal representing an original image,
ii) calculating a pixel vector at each of pixels in each of the band-limited images, which are represented by the band-limited image signals,
iii) separating a noise component and an edge component of each of the band-limited images in accordance with the calculated pixel vector,
iv) performing smoothing processing for the noise component and/or enhancement processing for the edge component on each of the band-limited image signals to obtain a processed band-limited image signal, and
v) obtaining a processed image signal in accordance with the thus obtained processed band-limited image signals.
In cases where a certain pixel in a band-limited image is taken as a pixel of interest, the pixel vector represents the direction and magnitude of the inclination of the pixel value of the pixel of interest. In order for the pixel vector to be calculated, for example, with respect to each of directions extending from the pixel of interest, the difference between the pixel value of the pixel of interest and the pixel value of a neighboring pixel (or the mean value of the pixel values of a plurality of pixels neighboring with the pixel of interest along a certain direction) may be calculated. The direction, which is associated with the largest difference value or the smallest difference value, may then be determined. The pixel vector may then be calculated from the difference value and the determined direction.
In cases where the direction associated with the largest difference value is taken as the direction of the pixel vector, the pixel vector represents the direction of the signal gradient. In cases where the direction associated with the smallest difference value is taken as the direction of the pixel vector, the pixel vector represents the direction of the equi-signal line.
In cases where the pixel vector is calculated along the direction of the signal gradient, if the difference between the pixel value of the pixel of interest and the pixel value of a neighboring pixel is taken as the magnitude of the pixel vector, a large pixel vector will represent that the pixel, for which the pixel vector has been calculated, is located at an edge component in the image. Also, in such cases, a small pixel vector will represent that the pixel, for which the pixel vector has been calculated, is located at a flat area in the image. Conversely, in cases where the pixel vector is calculated along the direction of the signal gradient, if the reciprocal of the difference between the pixel value of the pixel of interest and the pixel value of a neighboring pixel is taken as the magnitude of the pixel vector, a small pixel vector will represent that the pixel, for which the pixel vector has been calculated, is located at an edge component in the image. Also, in such cases, a large pixel vector will represent that the pixel, for which the pixel vector has been calculated, is located at a flat area in the image.
Further, in cases where the pixel vector is calculated along the direction of the equi-signal line, if the difference between the pixel value of the pixel of interest and the pixel value of a neighboring pixel is taken as the magnitude of the pixel vector, a small pixel vector will represent that the pixel, for which the pixel vector has been calculated, is located at an edge component in the image. Also, in such cases, a large pixel vector will represent that the pixel, for which the pixel vector has been calculated, is located at a flat area in the image. Conversely, in cases where the pixel vector is calculated along the direction of the equi-signal line, if the reciprocal of the difference between the pixel value of the pixel of interest and the pixel value of a neighboring pixel is taken as the magnitude of the pixel vector, a large pixel vector will represent that the pixel, for which the pixel vector has been calculated, is located at an edge component in the image. Also, in such cases, a small pixel vector will represent that the pixel, for which the pixel vector has been calculated, is located at a flat area in the image.
As the direction of the pixel vector, the direction associated with the largest difference value and the direction associated with the second largest difference value may be taken. Alternatively, as the direction of the pixel vector, the direction associated with the smallest difference value and the direction associated with the second smallest difference value may be taken. In such cases, the pixel vector is composed of two vectors.
As described above, in cases where the pixel vector is calculated along the direction of the equi-signal line with respect to a certain pixel of interest, and the reciprocal of the difference described above is taken as the magnitude of the pixel vector, it may be regarded that a large pixel vector represents that the pixel of interest is located at an edge in the image. Also, in such cases, it may be regarded that a small pixel vector represents that the pixel of interest is located at a flat image density area in the image. At the flat area in the image, the pixel of interest may be regarded as being noise.
From the foregoing, it can be found that, in cases where the noise component and the edge component of each of the band-limited images are to be separated in accordance with the calculated pixel vector, a judgment as to whether the pixel is located at the edge or the flat area may be made in accordance with the direction and/or the magnitude of the pixel vector, and the noise component and the edge component may be separated from the band-limited image signal in accordance with the results of the judgment.
The term xe2x80x9csmoothing processing for a noise componentxe2x80x9d as used herein means the processing for setting the pixel value of the pixel corresponding to the noise component at a small value. The term xe2x80x9cenhancement processing for an edge componentxe2x80x9d as used herein means the processing for setting the pixel value of the pixel corresponding to the edge component at a large value.
In the first image processing method in accordance with the present invention, the separation of the noise component and the edge component should preferably be performed in accordance with the pixel vector having been calculated for each pixel and a pixel vector at a pixel neighboring with each pixel.
Also, the first image processing method in accordance with the present invention should preferably be modified such that the pixel vector at a certain pixel in a band-limited image of a certain frequency band is corrected in accordance with the pixel vector at the pixel, which corresponds to the certain pixel, in a band-limited image of a frequency band lower than the certain frequency band (the correction technique will hereinbelow be referred to as the first correction technique), and
the separation of the noise component and the edge component is performed in accordance with the corrected pixel vector in lieu of the pixel vector at the certain pixel before being corrected.
The term xe2x80x9ccorrecting a pixel vectorxe2x80x9d as used herein for the first correction technique means that the direction of the pixel vector at the certain pixel in the band-limited image of the certain frequency band is corrected so as to coincide with the direction of the pixel vector at the pixel, which corresponds to the certain pixel, in the band-limited image of the frequency band lower than the certain frequency band. In cases where the pixel vector is thus corrected, correction may also be made with respect to the pixel vector at the neighboring pixel.
Further, the first image processing method in accordance with the present invention should preferably be modified such that a variance value in a predetermined region containing a certain pixel in a band-limited image of a certain frequency band is calculated,
a judgment as to whether the pixel vector at the certain pixel is to be corrected or may not be corrected is made in accordance with the variance value,
in cases where it has been judged that the pixel vector at the certain pixel is to be corrected, the pixel vector at the certain pixel is corrected in accordance with the pixel vector at the pixel, which corresponds to the certain pixel, in a band-limited image of a frequency band lower than the certain frequency band (the correction technique will hereinbelow be referred to as the second correction technique), and
the separation of the noise component and the edge component is performed in accordance with the corrected pixel vector in lieu of the pixel vector at the certain pixel before being corrected.
The variance value may be the variance value in the predetermined region or the difference value between the value of the pixel of interest, for which the pixel vector has been calculated, and the value of a neighboring pixel. Also, for example, in cases where the pixel vector has been calculated from the pixel of interest and eight neighboring pixels, the difference value described above may be the sum of the differences between the value of the pixel of interest and the values of the eight neighboring pixels, a mean value of the values of the differences, or the like.
As described above, a judgment as to whether the pixel vector at the certain pixel is to be corrected or may not be corrected is made in accordance with the variance value. Specifically, in cases where the variance value in the predetermined region containing the certain pixel is smaller than the variance value in the other region, it is judged that the predetermined region is a flat area, and that the band-limited image of the low frequency band need not be referred to. In cases where the variance value in the predetermined region containing the certain pixel is larger than the variance value in the other region, it is judged that the band-limited image of the low frequency band should be referred to.
The term xe2x80x9ccorrecting a pixel vectorxe2x80x9d as used herein for the second correction technique means that the direction of the pixel vector at the certain pixel in the band-limited image of the certain frequency band is corrected so as to coincide with the direction of the pixel vector at the pixel, which corresponds to the certain pixel, in the band-limited image of the frequency band lower than the certain frequency band. In cases where the pixel vector is thus corrected, correction may also be made with respect to the pixel vector at the neighboring pixel.
In the first image processing method in accordance with the present invention, as the techniques for forming the band-limited image signals, one of various techniques may be employed. For example, the band-limited image signals may be formed such that they represent the band-limited images of the original image size. By way of example, smoothing may be performed on the original image by utilizing masks of a plurality of different sizes, and a plurality of the band-limited image signals representing the band-limited images of the original image size may thereby be obtained. Also, after the noise component and the edge component in each of the band-limited images have been separated, the smoothing processing for the noise component and/or the enhancement processing for the edge component may be performed on each of the band-limited image signals.
Also, as a technique for forming the band-limited image signals, multi-resolution transform processing may be utilized. Specifically, multi-resolution transform processing may be performed on the original image signal to form the band-limited image signals, and inverse multi-resolution transform processing may be performed on the processed band-limited image signals to obtain the processed image signal. The inverse multi-resolution transform processing is the processing, which corresponds to the multi-resolution transform processing and with which the original signal can be restored (reversibly or irreversibly). In cases where the band-limited image signals are to be formed by performing the multi-resolution transform processing on the original image signal, for example, a technique may be employed, wherein the original image signal is transformed into signals, each of which has frequency response characteristics of one of plurality of different frequency bands, by utilizing Laplacian pyramid decomposition with the Laplacian pyramid technique or by utilizing the wavelet transform technique. In cases where the band-limited image signals have been obtained by utilizing the Laplacian pyramid decomposition, a Laplacian pyramid reconstruction technique is employed as the inverse multi-resolution transform processing. In cases where the band-limited image signals have been obtained by utilizing the wavelet transform technique, inverse wavelet transform processing is employed as the inverse multi-resolution transform processing.
In cases where the multi-resolution transform processing is thus utilized, when the band-limited image signals at the respective resolution levels are compared with one another, the frequency band of the image, which each band-limited image signal can express, becomes low for the image of a resolution lower than a certain resolution level (i.e., for the image of a low pixel density). Therefore, in cases where the multi-resolution transform processing is utilized, the aforesaid term xe2x80x9cfrequency band lower than a certain frequency bandxe2x80x9d becomes equivalent to the term xe2x80x9cresolution lower than a certain resolution level.xe2x80x9d Accordingly, for example, in cases where the pixel vector is to be corrected, the direction of the pixel vector at a certain pixel in the image of a certain resolution level may be corrected so as to coincide with the direction of the pixel vector at the pixel, which corresponds to the certain pixel, in the image of the resolution level lower than the certain resolution level.
The present invention also provides a second image processing method, comprising the steps of:
i) forming band-limited image signals representing images, each of which is of one of a plurality of different frequency bands, from an original image signal representing an original image,
ii) calculating a pixel vector at each of pixels in each of the band-limited images, which are represented by the band-limited image signals,
iii) smoothing each of the band-limited image signals in accordance with a direction of the calculated pixel vector to obtain a smoothed band-limited image signal, and
iv) obtaining a processed image signal in accordance with the thus obtained smoothed band-limited image signals.
The second image processing method in accordance with the present invention should preferably be modified such that a neighboring pixel vector at a pixel neighboring with a certain pixel is calculated, and the smoothing is performed in accordance with the direction of the pixel vector at the certain pixel and the direction of the neighboring pixel vector.
The term xe2x80x9csmoothing each band-limited image signal in accordance with a direction of a pixel vectorxe2x80x9d as used herein means that the band-limited image signal is smoothed in accordance with the direction of the pixel vector such that the edge component is kept and noise contained in the edge component (i.e., noise on the edge) is suppressed. For example, in cases where the pixel vector is the vector along the direction of the equi-signal line, the smoothing in accordance with the direction of the pixel vector may be performed by utilizing the pixel of interest, for which the pixel vector has been calculated, a pixel lying in the vector direction, and a pixel lying on the side reverse to the vector direction. Also, for the smoothing, a technique may be employed, where the mean value of the pixel value of the pixel of interest and the pixel value of the pixel lying in the direction of the pixel vector is calculated. Alternatively, a technique for performing the smoothing by utilizing a smoothing filter may be employed.
In order for the processed image signal to be obtained in accordance with the smoothed band-limited image signals, one of various techniques may be employed, wherein the noise components contained in the band-limited image signals before being smoothed are suppressed by the utilization of the smoothed band-limited image signals.
Also, the second image processing method in accordance with the present invention should preferably be modified such that the pixel vector is corrected by the utilization of the first or second correction technique described above, and the smoothing is performed in accordance with the direction of the corrected pixel vector.
Further, the second image processing method in accordance with the present invention should preferably be modified such that a noise component and an edge component of each of smoothed band-limited images, which are represented by the smoothed band-limited image signals, are separated in accordance with a magnitude of the pixel vector,
smoothing processing for the noise component and/or enhancement processing for the edge component is performed on each of the smoothed band-limited image signals to obtain a processed band-limited image signal, and
the processed image signal is obtained in accordance with the thus obtained processed band-limited image signals in lieu of the smoothed band-limited image signals.
In such cases, the second image processing method in accordance with the present invention should preferably be modified such that a neighboring pixel vector at a pixel neighboring with a certain pixel is calculated, and the separation of the noise component and the edge component is performed in accordance with the magnitude of the pixel vector at the certain pixel and the magnitude of the neighboring pixel vector.
As in the first image processing method in accordance with the present invention, the smoothing processing for the noise component is the processing for setting the pixel value of the pixel corresponding to the noise component at a small value. Also, the enhancement processing for the edge component is the processing for setting the pixel value of the pixel corresponding to the edge component at a large value.
Also, in such cases, the second image processing method in accordance with the present invention should preferably be modified such that the pixel vector is corrected by the utilization of the first or second correction technique described above, and the separation of the noise component and the edge component is performed in accordance with the magnitude of the corrected pixel vector.
In the second image processing method in accordance with the present invention, as in the first image processing method in accordance with the present invention, as the technique for forming the band-limited image signals, one of various techniques may be employed. For example, multi-resolution transform processing may be performed on the original image signal to form the band-limited image signals, predetermined processing may then be performed, and inverse multi-resolution transform processing may be performed on the smoothed band-limited image signals to obtain the processed image signal. Alternatively, in cases where the noise component and the edge component of each of the smoothed band-limited image signals are separated, inverse multi-resolution transform processing may be performed on the processed band-limited image signals to obtain the processed image signal.
The present invention further provides an apparatus for carrying out the first image processing method in accordance with the present invention. Specifically, the present invention further provides a first image processing apparatus, comprising:
i) band-limited image signal forming means for forming band-limited image signals representing images, each of which is of one of a plurality of different frequency bands, from an original image signal representing an original image,
ii) pixel vector calculating means for calculating a pixel vector at each of pixels in each of the band-limited images, which are represented by the band-limited image signals,
iii) separation means for separating a noise component and an edge component of each of the band-limited images in accordance with the calculated pixel vector,
iv) processing means for performing smoothing processing for the noise component and/or enhancement processing for the edge component on each of the band-limited image signals to obtain a processed band-limited image signal, and
v) image signal generating means for obtaining a processed image signal in accordance with the thus obtained processed band-limited image signals.
In the first image processing apparatus in accordance with the present invention, the separation means should preferably perform the separation of the noise component and the edge component in accordance with the pixel vector having been calculated for each pixel and a pixel vector at a pixel neighboring with each pixel.
Also, the first image processing apparatus in accordance with the present invention should preferably be modified such that the apparatus further comprises correction means for correcting the pixel vector at a certain pixel in a band-limited image of a certain frequency band in accordance with the pixel vector at the pixel, which corresponds to the certain pixel, in a band-limited image of a frequency band lower than the certain frequency band, and
the separation means separates the noise component and the edge component in accordance with the corrected pixel vector in lieu of the pixel vector at the certain pixel before being corrected.
Further, the first image processing apparatus in accordance with the present invention should preferably be modified such that the apparatus further comprises:
a) variance value calculating means for calculating a variance value in a predetermined region containing a certain pixel in a band-limited image of a certain frequency band,
b) judgment means for making a judgment as to whether the pixel vector at the certain pixel is to be corrected or may not be corrected, the judgment being made in accordance with the variance value, and
c) correction means for operating such that, in cases where it has been judged that the pixel vector at the certain pixel is to be corrected, the correction means corrects the pixel vector at the certain pixel in accordance with the pixel vector at the pixel, which corresponds to the certain pixel, in a band-limited image of a frequency band lower than the certain frequency band, and
the separation means separates the noise component and the edge component in accordance with the corrected pixel vector in lieu of the pixel vector at the certain pixel before being corrected.
Furthermore, the first image processing apparatus in accordance with the present invention may be modified such that the band-limited image signal forming means is provided with multi-resolution transform processing means for performing multi-resolution transform processing on the original image signal to form the band-limited image signals, and
the image signal generating means is provided with inverse multi-resolution transform processing means for performing inverse multi-resolution transform processing on the processed band-limited image signals to obtain the processed image signal.
The present invention still further provides an apparatus for carrying out the second image processing method in accordance with the present invention. Specifically, the present invention still further provides a second image processing apparatus, comprising:
i) band-limited image signal forming means for forming band-limited image signals representing images, each of which is of one of a plurality of different frequency bands, from an original image signal representing an original image,
ii) pixel vector calculating means for calculating a pixel vector at each of pixels in each of the band-limited images, which are represented by the band-limited image signals,
iii) smoothing means for smoothing each of the band-limited image signals in accordance with a direction of the calculated pixel vector to obtain a smoothed band-limited image signal, and
iv) image signal generating means for obtaining a processed image signal in accordance with the thus obtained smoothed band-limited image signals.
The second image processing apparatus in accordance with the present invention should preferably be modified such that the smoothing means calculates a neighboring pixel vector at a pixel neighboring with a certain pixel and performs the smoothing in accordance with the direction of the pixel vector at the certain pixel and the direction of the neighboring pixel vector.
Also, the second image processing apparatus in accordance with the present invention should preferably be modified such that the apparatus further comprises correction means for correcting the pixel vector at a certain pixel in a band-limited image of a certain frequency band in accordance with the pixel vector at the pixel, which corresponds to the certain pixel, in a band-limited image of a frequency band lower than the certain frequency band, and
the smoothing means performs the smoothing in accordance with the direction of the corrected pixel vector in lieu of the direction of the pixel vector at the certain pixel before being corrected.
Further, the second image processing apparatus in accordance with the present invention should preferably be modified such that the apparatus further comprises:
a) variance value calculating means for calculating a variance value in a predetermined region containing a certain pixel in a band-limited image of a certain frequency band,
b) judgment means for making a judgment as to whether the pixel vector at the certain pixel is to be corrected or may not be corrected, the judgment being made in accordance with the variance value, and
c) correction means for operating such that, in cases where it has been judged that the pixel vector at the certain pixel is to be corrected, the correction means corrects the pixel vector at the certain pixel in accordance with the pixel vector at the pixel, which corresponds to the certain pixel, in a band-limited image of a frequency band lower than the certain frequency band, and
the smoothing means performs the smoothing in accordance with the direction of the corrected pixel vector in lieu of the direction of the pixel vector at the certain pixel before being corrected.
Furthermore, the second image processing apparatus in accordance with the present invention should preferably be modified such that the apparatus further comprises:
separation means for separating a noise component and an edge component of each of smoothed band-limited images, which are represented by the smoothed band-limited image signals, in accordance with a magnitude of the pixel vector, and
processing means for performing smoothing processing for the noise component and/or enhancement processing for the edge component on each of the smoothed band-limited image signals to obtain a processed band-limited image signal, and
the image signal generating means obtains the processed image signal in accordance with the thus obtained processed band-limited image signals in lieu of the smoothed band-limited image signals.
In such cases, the second image processing apparatus in accordance with the present invention should preferably be modified such that the separation means separates the noise component and the edge component in accordance with the magnitude of the pixel vector at a certain pixel and the magnitude of a neighboring pixel vector at a pixel neighboring with the certain pixel.
Also, the second image processing apparatus in accordance with the present invention should preferably be modified such that the apparatus further comprises correction means for correcting the pixel vector at a certain pixel in a band-limited image of a certain frequency band in accordance with the pixel vector at the pixel, which corresponds to the certain pixel, in a band-limited image of a frequency band lower than the certain frequency band, and
the separation means separates the noise component and the edge component in accordance with the magnitude of the corrected pixel vector in lieu of the magnitude of the pixel vector at the certain pixel before being corrected.
Further, the second image processing apparatus in accordance with the present invention should preferably be modified such that the apparatus further comprises:
a) variance value calculating means for calculating a variance value in a predetermined region containing a certain pixel in a band-limited image of a certain frequency band,
b) judgment means for making a judgment as to whether the pixel vector at the certain pixel is to be corrected or may not be corrected, the judgment being made in accordance with the variance value, and
c) correction means for operating such that, in cases where it has been judged that the pixel vector at the certain pixel is to be corrected, the correction means corrects the pixel vector at the certain pixel in accordance with the pixel vector at the pixel, which corresponds to the certain pixel, in a band-limited image of a frequency band lower than the certain frequency band, and
the separation means separates the noise component and the edge component in accordance with the magnitude of the corrected pixel vector in lieu of the magnitude of the pixel vector at the certain pixel before being corrected.
Furthermore, the second image processing apparatus in accordance with the present invention may be modified such that the band-limited image signal forming means is provided with multi-resolution transform processing means for performing multi-resolution transform processing on the original image signal to form the band-limited image signals, and
the image signal generating means is provided with inverse multi-resolution transform processing means for performing inverse multi-resolution transform processing on the smoothed band-limited image signals to obtain the processed image signal.
Alternatively, the second image processing apparatus in accordance with the present invention, wherein the noise component and the edge component of each of the smoothed band-limited image signals are separated, may be modified such that the band-limited image signal forming means is provided with multi-resolution transform processing means for performing multi-resolution transform processing on the original image signal to form the band-limited image signals, and
the image signal generating means is provided with inverse multi-resolution transform processing means for performing inverse multi-resolution transform processing on the processed band-limited image signals to obtain the processed image signal.
The present invention also provides a recording medium, on which a program for causing a computer to execute the first, second, or third image processing method in accordance with the present invention has been recorded and from which the computer is capable of reading the program.
With the first image processing method and apparatus in accordance with the present invention, the pixel vector at each of pixels in each of the band-limited images is calculated. Also, the noise component and the edge component of each of the band-limited images are separated in accordance with the calculated pixel vector.
As described above, the magnitude of the pixel vector varies in accordance with whether the pixel vector is calculated along the direction of the equi-signal line or along the direction of the signal gradient, and whether the pixel vector is calculated as the value of the difference between the pixel value of the pixel of interest and the pixel value of the neighboring pixel or as the reciprocal of the difference. For example, the pixel vector may be calculated along the direction of the equi-signal line, and the reciprocal of the difference between the pixel value of the pixel of interest and the pixel value of the neighboring pixel may be taken as the magnitude of the pixel vector. In such cases, the pixel vector is large at the edge area in the image and is small at the flat area in the image. Therefore, the noise component and the edge component of each of the band-limited images can be separated in accordance with the magnitude of the pixel vector. Further, the smoothing processing for reducing the pixel value of the pixel corresponding to the separated noise component and/or the enhancement processing for enhancing the pixel value corresponding to the separated edge component is performed on each of the band-limited image signals to obtain the processed band-limited image signal. As a result, in each band-limited image, the noise component becomes imperceptible, and the edge component becomes perceptible. Accordingly, in the processed image signal obtained by performing the inverse multi-resolution transform processing on the thus obtained processed band-limited image signals, the edge component becomes perceptible, and the noise component becomes imperceptible. As a result, an image having good image quality can be reproduced from the processed image signal.
In cases where the pixel vector is calculated along the direction of the equi-signal line, and the reciprocal of the difference between the pixel value of the pixel of interest and the pixel value of the neighboring pixel is taken as the magnitude of the pixel vector, if the value of the pixel vector is comparatively small, it may be regarded that the pixel, for which the pixel vector has been calculated, is located at the flat area in the image, i.e. is located at the noise component in the image. However, in such cases, there is a probability that the pixel, for which the pixel vector has been calculated, will be located at a fine edge in the image. In cases where the pixel, for which the pixel vector has been calculated, is located at an edge in the image, the pixel vectors at the neighboring pixels are directed in the same direction. In cases where the pixel, for which the pixel vector has been calculated, is noise in the image, the pixel vectors at the neighboring pixels are directed in random directions. Therefore, with the first image processing method and apparatus in accordance with the present invention, wherein the separation of the noise component and the edge component is performed in accordance with the pixel vector having been calculated for each pixel and pixel vectors at pixels neighboring with each pixel, the probability that a certain pixel will represent an edge or noise can be enhanced. Accordingly, the noise component and the edge component can be separated more accurately.
Also, a comparatively large edge component contained in the original image remains in the image of a low frequency band, and a noise component contained in the original image becomes small in the image of the low frequency band. Therefore, the direction of the pixel vector at a certain pixel in a band-limited image of a certain frequency band should preferably be corrected so as to coincide with the direction of the pixel vector at the pixel, which corresponds to the certain pixel, in the band-limited image of a frequency band lower than the certain frequency band. In such cases, if the certain pixel is located at the edge component, the corrected pixel vector will represent the edge component more accurately. In the image of the low frequency band, a fine noise component becomes smaller than in the image of the high frequency band. Therefore, in cases where the certain pixel is located at the noise component, in the image of the low frequency band, the pixel vector is directed in a random direction, and the magnitude of the pixel vector becomes small. In such cases, the corrected pixel vector represents the flat area, i.e. represents the noise component, more accurately. Accordingly, with the first image processing method and apparatus in accordance with the present invention, wherein the pixel vector at the certain pixel in the band-limited image of the certain frequency band is corrected in accordance with the pixel vector at the pixel, which corresponds to the certain pixel, in the band-limited image of the frequency band lower than the certain frequency band, the probability that the certain pixel will represent the edge or noise can be enhanced, and the separation of the noise component and the edge component can be performed more accurately.
Further, in the image of a comparatively high frequency band, such as the image of a comparatively high resolution level obtained in cases where the multi-resolution transform processing is performed on the original image signal representing the original image, detail edge information is expressed. In the image of a middle frequency band, edge information of the middle frequency band is expressed. Also, in the image of a low frequency band, large edge information of the low frequency band is expressed. Ordinarily, the images of the respective frequency bands have the characteristics such that, as the frequency band becomes high, energy which the image of the high frequency band has becomes low, and such that energy of noise does not depend upon the frequency band. Therefore, as the frequency band becomes low, the image has a high signal-to-noise ratio. As for an area in the original image containing no noise (shown in FIG. 8A), a signal occurs only at an edge area in every band-limited image (as shown in FIGS. 8B, 8C, and 8D). Therefore, in cases where the variance value of pixel values in a predetermined region containing the pixel of interest, for which the pixel vector has been calculated, in the image of a comparatively high frequency band is small, even if the pixel vector at the corresponding pixel in the image of a low frequency band is not referred to, it can be regarded that the pixel of interest, for which the pixel vector has been calculated, is located at a flat area in the image.
As for an area in the original image containing noise (shown in FIG. 9A), in the image of the high frequency band (shown in FIG. 9B), the directions of the pixel vectors are disturbed by the effects of noise, and the variance value becomes large. However, as the frequency band becomes low (as shown in FIGS. 9C and 9D), the effects of noise upon the signal become small, and the variance value becomes small. Therefore, in cases where the variance value of pixel values in a predetermined region containing the pixel, for which the pixel vector has been calculated, in a certain band-limited image is large, if the pixel vector at the corresponding pixel in the image of a low frequency band is not referred to, it cannot be found whether the pixel, for which the pixel vector has been calculated, is located at a flat area or at an edge area in the image. Accordingly, in cases where the variance value described above is large in a certain band-limited image, the pixel vector at the pixel in the certain band-limited image should preferably be corrected by referring to the image of a low frequency band and causing the pixel vector to coincide with the pixel vector of the corresponding pixel in the image of the low frequency band. As a result, the pixel vector can be corrected such that the pixel vector at the flat area represents the flat area more accurately, and such that the pixel vector at the edge area represents the edge area more accurately. Therefore, in accordance with the corrected pixel vector, the noise component and the edge component can be separated accurately.
With the second image processing method and apparatus in accordance with the present invention, as in the first image processing method and apparatus in accordance with the present invention, the pixel vector at each of pixels in each of the band-limited images is calculated. Also, with the second image processing method and apparatus in accordance with the present invention, each of the band-limited images is smoothed in accordance with the direction of the calculated pixel vector.
In cases where noise is mixed in the original image, noise is also contained in the edge component in the image. However, with the second image processing method and apparatus in accordance with the present invention, each of the band-limited image signals is smoothed in accordance with the direction of the calculated pixel vector or the direction of the corrected pixel vector, which has been obtained with the first or second correction technique described above, and the smoothed band-limited image signal is thereby obtained. Also, the processed image signal is obtained in accordance with the thus obtained smoothed band-limited image signals. Therefore, noise on the edge can be suppressed such that the edge component may not be lost. Also, noise in the flat area other than the edge can be suppressed. Accordingly, ultimately, noise on the edge becomes imperceptible, and noise in the flat area becomes imperceptible.
Further, with the second image processing method and apparatus in accordance with the present invention, wherein, after the smoothing, the noise component and the edge component of each of smoothed band-limited images, which are represented by the smoothed band-limited image signals, are separated in accordance with the magnitude of the pixel vector, and the smoothing processing for the noise component and/or the enhancement processing for the edge component is performed on each of the smoothed band-limited image signals to obtain the processed band-limited image signal, the edge enhancement can be performed such that noise on the edge may not become perceptible. Also, noise in the flat area can be suppressed even further. Therefore, an image having image quality enhanced even further can be reproduced.